Image processing apparatus, image processing method, and storage medium

ABSTRACT

An image processing apparatus includes a generation unit configured to generate first region data indicating a region corresponding to a general uneven shape of each of a plurality of bundles included in a three-dimensional object and second region data indicating a region corresponding to an uneven shape of an extremely small substance included in each of the bundles by analyzing image data, and an output unit configured to output shape data after correcting the shape data by adding the general uneven shape of each of the plurality of bundles included in the three-dimensional object to the general uneven shape of the three-dimensional object and adding the uneven shape of the extremely small substance included in each of the bundles on the general uneven shape of each of the plurality of bundles included in the three-dimensional object based on the first region data and the second region data.

BACKGROUND Field of the Disclosure

The present disclosure relates to an image processing technique forreproducing an extremely small uneven shape on a surface of an object.

Description of the Related Art

In recent years, a technique for acquiring and reproducing athree-dimensional shape of a human head has been demanded for thepurpose of use in, for example, fabrication of a bust with use of athree-dimensional (3D) printer and computer graphics (CG) reproductionof a cosmetic makeup. As a technique for acquiring data expressing atexture of head hair, Japanese Patent No. 3832065 discusses that aportion corresponding to the head hair can be extracted from an imageand an undulation can be added to the extracted portion.

However, the technique discussed in Japanese Patent No. 3832065 replacesa head hair image expressing the portion corresponding to the head hairthat is extracted from the image with a preregistered fine hair pattern,and therefore cannot reproduce a hair bundle flowing through a widerange. Further, reproduction of fine unevenness with high accuracyrequires preparation of a large number of templates of hair patternsrespectively having finely changed straight-line directions.

SUMMARY

The present disclosure is directed to providing image processing forreproducing a fibrous uneven shape existing on a surface of an objectwith high accuracy.

According to an aspect of the present disclosure, an image processingapparatus is configured to output data indicating a shape of athree-dimensional object including a plurality of bundles each includinga plurality of extremely small substances. The image processingapparatus includes a first acquisition unit configured to acquire imagedata acquired by imaging the three-dimensional object, a secondacquisition unit configured to acquire shape data indicating a generaluneven shape of the three-dimensional object, a generation unitconfigured to generate first region data indicating a regioncorresponding to a general uneven shape of each of the plurality ofbundles included in the three-dimensional object and second region dataindicating a region corresponding to an uneven shape of the extremelysmall substance included in each of the bundles by analyzing the imagedata, and an output unit configured to output the shape data aftercorrecting the shape data by adding the general uneven shape of each ofthe plurality of bundles included in the three-dimensional object to thegeneral uneven shape of the three-dimensional object and adding theuneven shape of the extremely small substance included in each of thebundles on the general uneven shape of each of the plurality of bundlesincluded in the three-dimensional object based on the first region dataand the second region data.

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of head hairs.

FIG. 2 is a block diagram illustrating an example of a hardwareconfiguration of an image processing apparatus, according to one or moreembodiments of the subject disclosure.

FIG. 3 illustrates an example of a method for imaging a target objectfrom multiple viewpoints, according to one or more embodiments of thesubject disclosure.

FIG. 4 is a block diagram illustrating an example of a logicalconfiguration of the image processing apparatus, according to one ormore embodiments of the subject disclosure.

FIG. 5 is a flowchart illustrating a flow of processing performed by theimage processing apparatus, according to one or more embodiments of thesubject disclosure.

FIG. 6 illustrates an example of a processing result, according to oneor more embodiments of the subject disclosure.

FIG. 7 illustrates an example of the processing result, according to oneor more embodiments of the subject disclosure.

FIGS. 8A, 8B, and 8C each illustrate an example of an uneven shape,according to one or more embodiments of the subject disclosure.

DESCRIPTION OF THE EMBODIMENTS

In the following description, an exemplary embodiment of the presentdisclosure will be described with reference to the drawings. However,the exemplary embodiment that will be described below is not intended tolimit the present disclosure, and not all of combinations of featuresthat will be described in the present exemplary embodiment arenecessarily essential to a solution of the present disclosure. Similarconfigurations will be described while being identified by the samereference numeral.

In the following description, a first exemplary embodiment will bedescribed. FIG. 1 is a schematic view of head hairs. As illustrated inFIG. 1, a shape of a hair can be regarded as including two types ofunevenness, low-frequency unevenness (a hair bundle) and high-frequencyunevenness (one hair to several hairs). In other words, head hair is athree-dimensional object including a plurality of bundles each includinga plurality of extremely small substances (fibers). The presentexemplary embodiment will be described based on the following example. Aregion corresponding to the head hair in an image is regarded asincluding the low-frequency unevenness and the high-frequency unevenness(unevenness having a higher frequency than the low-frequencyunevenness), and their respective positions (regions) are identified.Then, an extremely small shape is added at a corresponding position(region) on three-dimensional shape data expressing a general shape ofthe head hair. The frequency of the unevenness is a frequency when theunevenness is deemed as a wave in which one dent and one bump forms onecycle.

FIG. 2 illustrates an example of a hardware configuration of an imageprocessing apparatus 1 according to the present exemplary embodiment.The image processing apparatus 1 is, for example, a computer, andincludes a central processing unit (CPU) 101, a read only memory (ROM)102, and a random access memory (RAM) 103. The CPU 101 executes anoperating system (OS) and various kinds of programs stored in the ROM102, a hard disk drive (HDD) 15, or the like with use of the RAM 103 asa work memory. Further, the CPU 101 controls each configuration via asystem bus 107. Processing according to a flowchart that will bedescribed below is performed by the CPU 101 while a program code storedin the ROM 102, the HDD 15, or the like is developed into the RAM 103.An input device 12, such as a mouse and a keyboard, and a printer 13 areconnected to a general-purpose interface (I/F) 104 via a serial bus 11.The HDD 15 and a general-purpose drive 16, which reads and writes datafrom and into various kinds of recording media, are connected to aSerial Advanced Technology Attachment (SATA) I/F 105 via a serial bus14. The CPU 101 uses the HDD 15 and the various kinds of recording mediamounted on the general-purpose drive 16 as locations to store variouskinds of data. A display 17 is connected to a video I/F 106. The CPU 101displays a user interface (UI) provided by the program on the display17, and receives an input such as a user instruction received via theinput device 12.

FIG. 4 is a block diagram illustrating a functional configuration of theimage processing apparatus 1 according to the present exemplaryembodiment. The image processing apparatus 1 includes an imageacquisition unit 301, a shape acquisition unit 302, a first generationunit 303, a second generation unit 304, and an output unit 305. Theimage acquisition unit 301 acquires a plurality of pieces of image dataacquired by imaging a target object from multiple viewpoints. The shapeacquisition unit 302 acquires the three-dimensional shape dataindicating a three-dimensional shape (a general uneven shape) of thetarget object (a subject) from the plurality of pieces of image data.The first generation unit 303 identifies the position of the regioncorresponding to the low-frequency unevenness in the regioncorresponding to the head hair in the image indicated by the image data,and generates first region data indicating the low-frequency unevenregion. Here, the low-frequency unevenness is a general uneven shape ofeach of the plurality of hair bundles included in the three-dimensionalobject. The second generation unit 304 identifies the position of theregion corresponding to the high-frequency unevenness in the regioncorresponding to the head hair in the image indicated by the image data,and generates second region data indicating the high-frequency unevenregion. Here, the high-frequency unevenness is an uneven shape of theextremely small substance (fiber) included in each of the hair bundles.The output unit 305 corrects the three-dimensional shape data by addingsemi-cylindrical unevenness at the region in the three-dimensional shapedata that corresponds to the region indicated by the first region data.Further, the output unit 305 corrects the three-dimensional shape databy adding semi-cylindrical unevenness at the region in thethree-dimensional shape data that corresponds to the region indicated bythe second region data. Then, the output unit 305 outputs the correctedthree-dimensional shape data to an external device, such as the HDD 15.

Next, the processing performed by the image processing apparatus 1 willbe described with reference to a flowchart illustrated in FIG. 5. TheCPU 101 executes the program stored in the ROM 102 with use of the RAM103 as the work memory, by which the image processing apparatus 1functions as each of the blocks illustrated in FIG. 4 and performsprocessing in each step illustrated in FIG. 5. The processing that willbe described below does not have to be entirely performed by the CPU101, and the image processing apparatus 1 may be configured in such amanner that a part or a whole of the processing is performed by one or aplurality of processing circuit(s) other than the CPU 101. In thefollowing description, each step (process) is indicated with “S” addedin front of a step number.

In step S401, the image acquisition unit 301 acquires the plurality ofpieces of image data that is a processing target from an externalstorage device 307 such as the HDD 15, and outputs the acquired data tothe shape acquisition unit 302, the first generation unit 303, and thesecond generation unit 304. The pieces of image data acquired here arean image group acquired by imaging a target object 204 from multipleviewpoints (a plurality of directions) with use of imaging devices 201to 203 as illustrated in FIG. 3. The plurality of pieces of image datais output to the shape acquisition unit 302 to acquire athree-dimensional shape of the target object 204. One piece of imagedata acquired by imaging the target object 204 from a front thereof(image data acquired from the imaging by the imaging device 202), amongthe plurality of pieces of image data, is output to the first generationunit 303 and the second generation unit 304. The image processingapparatus 1 may be configured to select the one piece of image data tobe output to the first generation unit 303 and the second generationunit 304 by receiving an instruction from the user.

In step S402, the shape acquisition unit 302 acquires thethree-dimensional shape data indicating a three-dimensional shape (ageneral shape of a hairstyle) of the target object 204 based on theplurality of pieces of image data input from the image acquisition unit301, and outputs the acquired three-dimensional shape data to the outputunit 305. One example of a method for generating the three-dimensionalshape from the image group acquired by the imaging from the multipleviewpoints is the known stereo matching. The stereo matching is atechnique that acquires a three-dimensional shape of an object with useof a plurality of images captured from different viewpoints. In thepresent exemplary embodiment, the three-dimensional shape is acquiredwith use of this stereo matching. The method for acquiring thethree-dimensional shape is not limited to the above-described stereomatching, and the shape acquisition unit 302 may directly acquirethree-dimensional shape data acquired from, for example, measurementwith use of a laser scanner or a depth sensor, or computer graphics (CG)rendering. Further, the shape acquisition unit 302 may acquire thethree-dimensional shape data by generating the three-dimensional shapedata in advance by the above-described method and storing the generatedthree-dimensional shape data into the external storage device 307 suchas the HDD 15 without generating the three-dimensional shape data in thepresent step. Generally, the three-dimensional shape data generated withan attempt to acquire a detailed shape such as hairs contains a largeamount of noise, and therefore should be subjected to noise reductionprocessing (smoothing processing). However, this noise reductionprocessing results in an undesirable loss of the detailed shape likehairs. On the other hand, the three-dimensional shape data generatedwith an attempt to acquire a rough shape does not contain the detailedshape like hairs. The three-dimensional shape data according to thepresent exemplary embodiment is data generated by the above-describedmethod, and data indicating a general uneven shape that does not containthe low-frequency unevenness (the hair bundle) and the high-frequencyunevenness (one hair to several hairs).

In step S403, the first generation unit 303 identifies the position ofthe low-frequency unevenness by analyzing the image data input from theimage acquisition unit 301. In the present exemplary embodiment, thefirst generation unit 303 identifies the position of the low-frequencyunevenness in the image data by detecting an edge (a change in aluminance) after applying a smoothing filter to the image data. Toidentify the position of the low-frequency unevenness, the firstgeneration unit 303 first smooths the image to facilitate identificationof a position of low-frequency unevenness that is approximately 5 mm to1 cm in actual dimension. In the present exemplary embodiment, aGaussian filter is applied to the image data as the processing forsmoothing the image (reducing a resolution of the image). Assuming thatI represents a luminance value in the image data, o_(g) represents astandard deviation, and 2w+1 is a filter size, an output luminance valueI_(g) when the Gaussian filter is applied to a pixel (u, v) is expressedby an equation (1).

$\begin{matrix}{{I_{g}\left( {u,v,w} \right)} = {\frac{1}{2{\pi\sigma}_{g}^{2}}{\sum\limits_{n = {- w}}^{W}{\sum\limits_{m = {- w}}^{W}{{I\left( {{u + m},{v + n}} \right)}{\exp \left( {- \frac{m^{2} + n^{2}}{2\sigma_{g}^{2}}} \right)}}}}}} & (1)\end{matrix}$

In this equation (1), the standard deviation σ_(g) corresponds to acycle of the low-frequency unevenness. If the data acquired by the imageacquisition unit 301 is image data indicating a color signal(Red-Green-Blue (RGB) value), the data is converted into grayscale imagedata indicating the luminance with use of a known conversion equation inadvance before the Gaussian filter is applied thereto. The acquisitionof the grayscale image data may be achieved by preparing the grayscaleimage data in the external storage device 307 in advance, and acquiringit by the image acquisition unit 301.

A moving average filter, a medial filter, or other filter may be usedinstead of the above-described Gaussian filter. An output luminancevalue I_(A) in the case where the moving average filter is used isexpressed by an equation (2).

$\begin{matrix}{{I_{A}\left( {u,v,w^{\prime}} \right)} = {\frac{1}{\left( {{2W^{\prime}} + 1} \right)^{2}}{\sum\limits_{n = {- w}}^{W}{\sum\limits_{m = {- w}}^{W}{I\left( {{u + m},{v + n}} \right)}}}}} & (2)\end{matrix}$

Further, the medial filter is a filter that re-sorts luminance values ofa pixel of interest and a region around it in descending order, andoutputs a median value when the luminance values are re-sorted.

Next, the first generation unit 303 convolves an unsharp mask on theimage data with the Gaussian filter applied thereto to identify theposition of the low-frequency unevenness. The unsharp mask is one offilters for emphasizing a color difference between pixels. A luminancevalue I′ of the output image when the unsharp mask is convolved isexpressed by an equation (3), assuming that 2w′+1 is a filter size and krepresents a sharpening constant. Normally, the filter size of theunsharp mask approximately matches the filter size of the Gaussianfilter.

I′(u, v, w′)=I(u, v)+k(I(u, v)−I _(g)(u, v, w′))   (3)

The edge may be detected (the position of the unevenness may beidentified) with use of a Gabor filter instead of the unsharp mask.

Lastly, the first generation unit 303 performs binarization processingby an equation (4) on the image data with the unsharp mask convolvedthereon, assuming that t represents a threshold value regarding theluminance. By this operation, the first generation unit 303 generatesthe first region data (binary data) indicating the position of thelow-frequency uneven region (a position corresponding to I″=1)

$\begin{matrix}{{I^{''}\left( {u,v} \right)} = \left\{ \begin{matrix}1 & {{{for}\mspace{14mu} {I^{\prime}\left( {u,v} \right)}} \geq t} \\0 & {{{for}\mspace{14mu} {I^{\prime}\left( {u,v} \right)}} < t}\end{matrix} \right.} & (4)\end{matrix}$

In step S404, the second generation unit 304 identifies the position ofthe high-frequency unevenness (a cycle: approximately 1 mm to 5 mm) byanalyzing the image data output from the image acquisition unit 301. Inthe present exemplary embodiment, the second generation unit 304identifies the position of the high-frequency unevenness in the imagedata by detecting the edge without applying the smoothing filter to theimage data. The second generation unit 304 performs the binarizationprocessing similarly to the first generation unit 303 after directlyconvolving the unsharp mask on the image data, to identify the positionof the high-frequency unevenness. The second generation unit 304generates the second region data (binary data) indicating the positionof the high-frequency uneven region based on a result of theidentification.

In step S405, the output unit 305 adds the low-frequency unevenness atthe position on the three-dimensional shape data that corresponds to theposition of the low-frequency unevenness, which is identified by thefirst generation unit 303, based on the first region data. In thepresent exemplary embodiment, a shape of the added predeterminedunevenness is assumed to be a semi-cylindrical shape, like an exampleillustrated in FIG. 8A. The shape of the unevenness is not limited tothe cylindrical shape, and may be a rod-like shape trapezoidal orrectangular in cross section, like examples illustrated in FIGS. 8B and8C. A method for correcting the three-dimensional shape data to add thelow-frequency unevenness will be described below.

In step S406, the output unit 305 adds the high-frequency unevenness atthe position on the three-dimensional shape data that corresponds to theposition of the high-frequency unevenness, which is identified by thesecond generation unit 304. A shape of the added predeterminedunevenness is assumed to be a semi-cylindrical shape but may be arod-like shape trapezoidal or rectangular in cross section, similarly tostep S405. Then, the cycle of the high-frequency unevenness added instep S406 is shorter than the cycle of the low-frequency unevenness. Amethod for correcting the three-dimensional shape data to add thehigh-frequency unevenness will be described below.

In the following description, the processing for correcting thethree-dimensional shape data to add the low-frequency unevenness (stepS405) will be described. In the present exemplary embodiment, assumethat the three-dimensional shape data indicating the general shape ofthe hairstyle, which is acquired from the stereo matching, is depthdata. The depth data is data indicating a distance (a depth) from thesubject to the imaging apparatus. The three-dimensional shape data isnot limited to the depth data, and may be mesh data, point group data,or other data. Alternatively, the three-dimensional shape data may bedata indicating a height from a surface of the image.

First, a normal vector n (n, v) for each pixel is calculated by anequation (5) with respect to the depth data having a depth d (u, v) ateach pixel.

n(u, v)=∇d(u, v)   (5)

In this equation (5), V represents a gradient. Next, a normal vectorcorresponding to a pixel having a pixel value of 1 in the binary dataacquired in step S403 is converted. The pixel having the pixel value of1 is searched for horizontally row by row from an upper left pixel to alower right pixel in the binary data I″. Then, based on the normalvector (u, v) of a pixel located at a center in a group of horizontallyconnected pixels having the pixel value of 1, the normal vectorcorresponding to each of the pixels in the pixel group is converted. Atthis time, the normal vector is converted by an equation (6), assumingthat c represents the number of connected pixels, (u′, v′) representsthe central pixel, and Re represents a three-dimensional rotation matrixfor a rotation by θ around a y axis.

$\begin{matrix}{{n^{\prime}\left( {u,v^{\prime}} \right)} = {R_{\frac{\pi {({u - u^{\prime}})}}{c}}{n\left( {u^{\prime},v^{\prime}} \right)}}} & (6)\end{matrix}$

Regarding the pixel for which the normal vector is converted, the depthis calculated from the normal vector by an equation (7), and the valueof the depth is replaced therewith.

$\begin{matrix}{{d\left( {u,v} \right)} = {{\sum\limits_{m = 0}^{u - 1}\frac{n_{x}^{\prime}\left( {m,v} \right)}{n_{z}^{\prime}\left( {m,v} \right)}} + {\sum\limits_{n = 0}^{v - 1}\frac{n_{x}^{\prime}\left( {u,n} \right)}{n_{z}^{\prime}\left( {u,n} \right)}}}} & (7)\end{matrix}$

The depth of the region at which the unevenness is added is corrected inthis manner, by which the low-frequency unevenness is reproduced on thethree-dimensional shape data.

Next, the processing for correcting the three-dimensional shape data toadd the high-frequency unevenness (step S406) will be described. Becausethe high-frequency unevenness is added on the low-frequency unevennessin a superimposed manner, the second region data indicating thehigh-frequency uneven region is corrected with use of the first regiondata indicating the low-frequency uneven region before thehigh-frequency unevenness is added. Regarding the correction, 0 is setto a pixel value of a pixel in the second region data indicating thehigh-frequency uneven region that corresponds to a pixel having a pixelvalue of 0 in the first region data indicating the low-frequency unevenregion. This setting prevents the high-frequency unevenness from beingadded at a position where the low-frequency unevenness is not added.After the correction of the second region data, the normal vector iscalculated by the equation (5), and then a normal vector correspondingto the position of the high-frequency unevenness is converted by theequation (6), similarly to step S405. Then, the depth of the pixel forwhich the normal vector is converted is calculated from the equation (7)and the value of the depth is replaced therewith.

In this manner, the image processing apparatus 1 according to thepresent exemplary embodiment detects the edge in the image data with thesmoothing filter applied thereto and the image data without thesmoothing filter applied thereto, and identifies the position of thelow-frequency unevenness and the position of the high-frequencyunevenness. Then, the image processing apparatus 1 according to thepresent exemplary embodiment reproduces the extremely small unevennessof the head hair by correcting the depth of the position indicated bythe three-dimensional shape data that corresponds to each of theidentified positions. FIG. 6 illustrates a shape acquired by imaging awig (the head hair) as the object having the fibrous unevenness on thesurface thereof and reproducing this wig from the imaged data, to makean effect from the present exemplary embodiment easily understandable.It can be confirmed from FIG. 6 that both the low-frequency shape andthe high-frequency shape are able to be reproduced. Further, thisprocessing can also be applied to a cloth having a fine fibrousstructure. FIG. 7 illustrates a shape acquired from a photographed knitcap. It can be confirmed from FIG. 7 that both the low-frequency shapeand the high-frequency shape are also able to be reproduced with respectto the knit cap similarly to hair. In this manner, according to thepresent exemplary embodiment, a shape of a subject having a similarmicrostructure to hair can also be reproduced.

[Exemplary Modification]

In the above-described exemplary embodiment, the extremely small unevenshape of the head hair is reproduced on the three-dimensional shapedata, but the uneven shape may also be reproduced by controlling aheight of a surface of an image indicated by two-dimensional image data.In this case, the image acquisition unit 301 acquires one piece of imagedata. The image acquisition unit 301 outputs the image data to the firstgeneration unit 303 and the second generation unit 304. The shapeacquisition unit 302 acquires two-dimensional image data having a heightor a depth from a reference surface at each pixel from the externalstorage device 307. Then, the height or the depth at the pixelcorresponding to each of the positions identified by the firstgeneration unit 303 and the second generation unit 304 is corrected. Theextremely small uneven shape of the head hair can be reproduced on thetwo-dimensional image data by the above-described processing.

In the above-described exemplary embodiment, the extremely small unevenshape of the head hair is reproduced on the three-dimensional shapedata, but the extremely small uneven shape of the head hair may bereproduced through a printed product by outputting the three-dimensionalshape data (or the two-dimensional image data) to a printer. In thiscase, a 3D printer or a printer capable of forming unevenness on asurface of a recording medium can be used as the printer to which thedata is output.

In the above-described exemplary embodiment, the unevenness is added tothe three-dimensional shape data with use of the region data identifyingthe uneven region without any correction made thereto. However, theregion data indicating the identified uneven region may contain noise,and therefore the unevenness may be added after this noise is reduced.In this case, the noise is reduced before the normal vector is convertedby the equation (6). One example of a method for reducing the noise is amethod using the number of connected pixels around the pixel ofinterest. If a threshold value t′ is set with respect to the number ofconnected pixels adjacent to the pixel of interest, the binarizationprocessing is provided as indicated by an equation (8). The equation (8)is an equation prepared to perform processing that, if the number ofconnected pixels is smaller than the threshold value t′, regards thispixel value as noise.

$\begin{matrix}{{I_{t}\left( {u,v} \right)} = \left\{ \begin{matrix}1 & {{{for}\mspace{14mu} {\sum\limits_{n = {- 1}}^{1}{\sum\limits_{m = {- 1}}^{1}{I\left( {{u + m},{v + n}} \right)}}}} \geq t^{\prime}} \\0 & {{{for}\mspace{14mu} {\sum\limits_{n = {- 1}}^{1}{\sum\limits_{m = {- 1}}^{1}{I\left( {{u + m},{v + n}} \right)}}}} < t^{\prime}}\end{matrix} \right.} & (8)\end{matrix}$

The noise reduction is not limited to the above-described one example,and the noise may be reduced with use of a median filter.

In the above-described exemplary embodiment, the normal vector isconverted with use of the equation (6), but the conversion of the normalvector is not limited to the above-described one example. The normalvector may be converted so as to reduce the number of connected pixelscorresponding to a width of the unevenness (so as to make across-sectional diameter of the added unevenness narrower than that ofthe identified position) to add further sharp unevenness. Assuming thatr represents the number of pixels by which the number of connectedpixels is reduced, the conversion of the normal vector is provided asindicated by an equation (9).

$\begin{matrix}{{n^{\prime}\left( {u,v^{\prime}} \right)} = {R_{\frac{\pi {({u - u^{\prime}})}}{{2c} - r}}{n\left( {u^{\prime},v^{\prime}} \right)}}} & (9)\end{matrix}$

In the above-described exemplary embodiment, the shape data is correctedby adding the unevenness having the predetermined certain shape at thecorresponding region in the shape data, but the present exemplaryembodiment is not limited thereto. For example, the present exemplaryembodiment may be configured in the following manner. A size of thegeneral uneven shape of each of the plurality of bundles included in thethree-dimensional object (the low-frequency unevenness) and a size ofthe uneven shape of the extremely small substance included in each ofthe bundles (the high-frequency unevenness) are identified based on theacquired image data. Then, the above-described shape data is correctedby adding an uneven shape of a size according to each of the identifiedsizes to the general uneven shape of the above-describedthree-dimensional object.

In the above-described exemplary embodiment, the pixels to be connectedare searched for horizontally in the binary data, but may be searchedfor vertically.

According to the present disclosure, the fibrous uneven shape existingon the surface of the object can be reproduced with high accuracy.

Other Embodiments

Embodiment(s) of the present disclosure can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc(BD)TM), a flash memory device, a memory card, and the like.

While the present disclosure has been described with reference toexemplary embodiments, it is to be understood that the disclosure is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2016-188407, filed Sep. 27, 2016, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing apparatus configured tooutput data indicating a shape of a three-dimensional object including aplurality of bundles each including a plurality of extremely smallsubstances, the image processing apparatus comprising: a firstacquisition unit configured to acquire image data acquired by imagingthe three-dimensional object; a second acquisition unit configured toacquire shape data indicating a general uneven shape of thethree-dimensional object; a generation unit configured to generate firstregion data indicating a region corresponding to a general uneven shapeof each of the plurality of bundles included in the three-dimensionalobject and second region data indicating a region corresponding to anuneven shape of the extremely small substance included in each of thebundles by analyzing the image data; and an output unit configured tooutput the shape data after correcting the shape data by adding thegeneral uneven shape of each of the plurality of bundles included in thethree-dimensional object to the general uneven shape of thethree-dimensional object and adding the uneven shape of the extremelysmall substance included in each of the bundles on the general unevenshape of each of the plurality of bundles included in thethree-dimensional object based on the first region data and the secondregion data.
 2. The image processing apparatus according to claim 1,wherein the first acquisition unit acquires a plurality of pieces ofimage data acquired by imaging the three-dimensional object from aplurality of directions, and wherein the second acquisition unitgenerates and acquires the shape data based on the plurality of piecesof image data.
 3. The image processing apparatus according to claim 1,wherein the shape data is data acquired from measurement with use of alaser scanner or a depth sensor, or from CG rendering.
 4. The imageprocessing apparatus according to claim 1, wherein the shape data istwo-dimensional image data indicating a height or a depth of thethree-dimensional object from a reference surface as the general unevenshape of the three-dimensional object.
 5. The image processing apparatusaccording to claim 1, wherein the generation unit identifies the regioncorresponding to the general uneven shape of each of the plurality ofbundles included in the three-dimensional object by detecting a changein a luminance in the image data after smoothing the image data, andidentifies the region corresponding to the uneven shape of the extremelysmall substance included in each of the bundles by detecting the changein the luminance in the image data without smoothing the image data. 6.The image processing apparatus according to claim 5, wherein thegeneration unit identifies the region corresponding to the generaluneven shape of each of the plurality of bundles included in thethree-dimensional object after smoothing the image data with use of aGaussian filter.
 7. The image processing apparatus according to claim 5,wherein the generation unit identifies the region corresponding to thegeneral uneven shape of each of the plurality of bundles included in thethree-dimensional object after smoothing the image data with use of amoving average filter or a median filter.
 8. The image processingapparatus according to claim 5, wherein the generation unit detects thechange in the luminance with use of an unsharp mask.
 9. The imageprocessing apparatus according to claim 5, wherein the generation unitdetects the change in the luminance with use of a Gabor filter.
 10. Theimage processing apparatus according to claim 1, wherein the shape datais data indicating a depth, wherein the image processing apparatusfurther comprises a calculation unit configured to calculate a normalvector on a surface of the three-dimensional object that corresponds tothe depth indicated by the shape data, and a conversion unit configuredto convert the normal vector in such a manner that a predetermined shapeis acquired as a shape of a region in the shape data that corresponds toeach of the region corresponding to the general uneven shape of each ofthe plurality of bundles included in the three-dimensional object, whichis indicated by the first region data, and the region corresponding tothe uneven shape of the extremely small substance included in each ofthe bundles, which is indicated by the second region data, and whereinthe output unit corrects the shape data by calculating a depth from thenormal vector converted by the conversion unit and replacing the depthindicated by the shape data with the calculated depth.
 11. The imageprocessing apparatus according to claim 10, wherein the predeterminedshape is a semi-cylindrical shape.
 12. The image processing apparatusaccording to claim 10, wherein the predetermined shape is a rod-likeshape trapezoidal or rectangular in cross section.
 13. The imageprocessing apparatus according to claim 1, wherein the output unitcorrects the shape data by adding an uneven shape having a sizeaccording to each of a size of the general uneven shape of each of theplurality of bundles included in the three-dimensional object and a sizeof the uneven shape of the extremely small substance included in each ofthe bundles, which are identified based on the image data, to thegeneral uneven shape of the three-dimensional object.
 14. The imageprocessing apparatus according to claim 1, wherein the three-dimensionalobject is head hair.
 15. The image processing apparatus according toclaim 1, wherein the three-dimensional object is a cloth.
 16. The imageprocessing apparatus according to claim 1, wherein the general unevenshape of each of the plurality of bundles included in thethree-dimensional object includes unevenness having a cycle of 5 mm to 1cm, and the uneven shape of the extremely small substance included ineach of the bundles includes unevenness having a cycle of 1 mm to 5 mm.17. The image processing apparatus according to claim 1, wherein theextremely small substance included in each of the bundles is anextremely small fiber included in each of the bundles.
 18. The imageprocessing apparatus according to claim 1, further comprising areduction unit configured to reduce noise contained in the first regiondata and the second region data, wherein the output unit corrects theshape data based on the first region data and the second region datawith the noise reduced by the reduction unit therein.
 19. The imageprocessing apparatus according to claim 10, wherein the conversion unitconverts the normal vector so as to make a diameter of a cross sectionof the predetermined shape narrower than the region in the shape datathat corresponds to each of the region corresponding to the generaluneven shape of each of the plurality of bundles included in thethree-dimensional object, which is indicated by the first region data,and the region corresponding to the uneven shape of the extremely smallsubstance included in each of the bundles, which is indicated by thesecond region data.
 20. An image processing method for outputting dataindicating a shape of a three-dimensional object including a pluralityof bundles each including a plurality of extremely small substances, theimage processing method comprising: acquiring image data acquired byimaging the three-dimensional object, as first acquisition; acquiringshape data indicating a general uneven shape of the three-dimensionalobject, as second acquisition; generating first region data indicating aregion corresponding to a general uneven shape of each of the pluralityof bundles included in the three-dimensional object and second regiondata indicating a region corresponding to an uneven shape of theextremely small substance included in each of the bundles by analyzingthe image data; and outputting the shape data after correcting the shapedata by adding the general uneven shape of each of the plurality ofbundles included in the three-dimensional object to the general unevenshape of the three-dimensional object and adding the uneven shape of theextremely small substance included in each of the bundles on the generaluneven shape of each of the plurality of bundles included in thethree-dimensional object based on the first region data and the secondregion data.
 21. A non-transitory computer-readable storage mediumstoring instructions that, when executed by a computer, cause thecomputer to perform a method comprising: acquiring image data acquiredby imaging a three-dimensional object including a plurality of bundleseach including a plurality of extremely small substances, as firstacquisition; acquiring shape data indicating a general uneven shape ofthe three-dimensional object, as second acquisition; generating firstregion data indicating a region corresponding to a general uneven shapeof each of the plurality of bundles included in the three-dimensionalobject and second region data indicating a region corresponding to anuneven shape of the extremely small substance included in each of thebundles by analyzing the image data; and outputting the shape data aftercorrecting the shape data by adding the general uneven shape of each ofthe plurality of bundles included in the three-dimensional object to thegeneral uneven shape of the three-dimensional object and adding theuneven shape of the extremely small substance included in each of thebundles on the general uneven shape of each of the plurality of bundlesincluded in the three-dimensional object based on the first region dataand the second region data.